Communication-Efficient Distributed SGD With Compressed Sensing

We consider large scale distributed optimization over a set of edge devices connected to central server, where the limited communication bandwidth between server and imposes significant bottleneck for procedure. Inspired by recent advances in federated learning, we propose stochastic gradient descent (SGD) type algorithm that exploits sparsity gradient, when possible, reduce burden. At heart is use compressed sensing techniques compression local gradients at device side; side, sparse approximation global recovered from noisy aggregated gradients. conduct theoretical analysis on convergence our presence noise perturbation incurred channels, also numerical experiments corroborate its effectiveness.